Information processing apparatus and information processing method

ABSTRACT

An information processing apparatus including an acquisition section that acquires an operation model indicating an operation pattern related to a target object, a calculation section that calculates an overall operation including an identified partial operation of operations of the target object, with reference to the operation model acquired by the acquisition section, and an output control section that outputs information indicating the overall operation of the target object that has been calculated by the calculation section.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is a U.S. National Phase of International PatentApplication No. PCT/JP2017/010517 filed on Mar. 15 2017, which claimspriority benefit of Japanese Patent Application No. JP 2016-119688 filedin the Japan Patent Office on Jun. 16, 2016. Each of theabove-referenced applications is hereby incorporated herein by referencein its entirety.

TECHNICAL FIELD

The present disclosure relates to an information processing apparatus,an information processing method, and a storage medium.

BACKGROUND ART

Recently, technology to visualize (that is, to digitize) a move of abody has been actively developed. In the field of sports, for example,technology is being developed to attach sensor devices to various partsof a body for visualizing a move of the body on the basis of themeasurement results and contributing to improvement in forms, and thelike.

For example, Patent Literature 1 described below discloses a technologyof appropriately setting an analysis section of output data obtainedfrom a sensor device mounted on a user or a tool used by a user, forenhancing determination accuracy of a motion pattern such as a serve anda smash in tennis.

CITATION LIST Patent Literature

Patent Literature 1: JP 2016-10714A

DISCLOSURE OF INVENTION Technical Problem

Nevertheless, it is hard to say that the technology proposed in PatentLiterature 1 described above is sufficient as a technology to visualizean operation of a target object. For example, one aspect of theinsufficiency lies on the reduction of an information amount requiredfor visualizing the operation of the target object. In view of theforegoing, a mechanism that can visualize an operation of a targetobject from a smaller amount of information is desirably provided.

Solution to Problem

According to the present disclosure, there is provided an informationprocessing apparatus including: an acquisition section configured toacquire an operation model indicating an operation pattern related to atarget object; a calculation section configured to calculate an overalloperation including an identified partial operation of operations of thetarget object, with reference to the operation model acquired by theacquisition section; and an output control section configured to outputoutput information indicating the overall operation of the target objectthat has been calculated by the calculation section.

In addition, according to the present disclosure, there is provided aninformation processing method including: acquiring an operation modelindicating an operation pattern related to a target object; calculating,by a processor, an overall operation including an identified partialoperation of operations of the target object, with reference to theacquired operation model; and outputting output information indicatingthe calculated overall operation of the target object.

In addition, according to the present disclosure, there is provided astorage medium storing a program for causing a computer to function as:an acquisition section configured to acquire an operation modelindicating an operation pattern related to a target object; acalculation section configured to calculate an overall operationincluding an identified partial operation of operations of the targetobject, with reference to the operation model acquired by theacquisition section; and an output control section configured to outputoutput information indicating the overall operation of the target objectthat has been calculated by the calculation section.

Advantageous Effects of Invention

As described above, according to the present disclosure, a mechanismthat can visualize an operation of a target object from a smaller amountof information is provided. Note that the effects described above arenot necessarily limitative. With or in the place of the above effects,there may be achieved any one of the effects described in thisspecification or other effects that may be grasped from thisspecification.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a diagram for describing an outline of a system according to afirst embodiment.

FIG. 2 is a block diagram illustrating an example of a configuration ofa sensor device according to the embodiment.

FIG. 3 is a block diagram illustrating an example of a configuration ofan information processing apparatus according to the embodiment.

FIG. 4 is a diagram illustrating an example of visualization processingof a user operation according to the embodiment.

FIG. 5 is a diagram illustrating an example of visualization processingof a user operation according to the embodiment.

FIG. 6 is a diagram illustrating an example of visualization processingof a user operation according to the embodiment.

FIG. 7 is a diagram for describing an example of a prediction algorithmaccording to the embodiment.

FIG. 8 is a diagram illustrating an example of a dimension-compressedoperation model according to the embodiment.

FIG. 9 is a diagram for describing an example of a prediction algorithmaccording to the embodiment.

FIG. 10 is a diagram for describing an example of a UI according to theembodiment.

FIG. 11 is a flowchart illustrating an example of a flow of operationvisualization processing executed in the information processingapparatus according to the embodiment.

FIG. 12 is a block diagram illustrating an example of a configuration ofan information processing apparatus according to a second embodiment.

FIG. 13 is a diagram illustrating an example of a UI according to theembodiment.

FIG. 14 is a flowchart illustrating an example of a flow of operationvisualization processing executed in the information processingapparatus according to the embodiment.

FIG. 15 is a block diagram illustrating an example of the hardwareconfiguration of the information processing apparatus according to eachof the embodiments.

MODE(S) FOR CARRYING OUT THE INVENTION

Hereinafter, (a) preferred embodiment(s) of the present disclosure willbe described in detail with reference to the appended drawings. Notethat, in this specification and the appended drawings, structuralelements that have substantially the same function and structure aredenoted with the same reference numerals, and repeated explanation ofthese structural elements is omitted.

In addition, in the specification and the drawings, differentalphabetical letters may be given to components having substantially thesame functional configuration for distinction after the same symbol isgiven to the components. For example, a plurality of components havingsubstantially the same functional configuration are distinguished assensor devices 10A, 10B, and 10C as necessary. However, in a case whereit is unnecessary to particularly distinguish each of the plurality ofcomponents having substantially the same functional configuration, onlythe same symbol is given. For example, in a case where it is unnecessaryto particularly distinguish sensor devices 10A, 10B, and 10C, the sensordevices are simply referred to as a sensor device 10.

Note that the description will be given in the following order.

1. First Embodiment

-   -   1.1. Outline of system    -   1.2. Configuration example of sensor device    -   1.3. Configuration example of information processing apparatus    -   1.4. Technical feature    -   1.5. Flow of processing

2. Second Embodiment

-   -   2.1. Configuration example of information processing apparatus    -   2.2. Technical feature    -   2.3. Flow of processing

3. Hardware configuration example

4. Conclusion

1. First Embodiment

The present embodiment is a mode of visualizing an operation of a realobject.

<1.1. Outline of System>

FIG. 1 is a diagram for describing an outline of a system 1 according tothe present embodiment. As illustrated in FIG. 1, the system 1 includesa plurality of sensor devices 10 (that is, 10A to 10C) attached to asensor attachment apparatus 20.

The sensor device 10 is a device that measures various kinds of data.The sensor device 10 is attached to a sensor attachment tool 21 includedin the sensor attachment apparatus 20 to perform measuring targeting amove of a target object. A target object may be a human, a dog, a cat,or other living organisms, or may be a non-living organism such as arobot. In the example illustrated in FIG. 1, a target object is a user(that is, a human). In addition, the target object may be an object tobe used by a living organism. For example, the target object may be atool to be used for games such as a golf club, a tennis racket, a skiboard, a ski boot, a goal, or a bat. In addition, the target object maybe a tool to be used for living such as an artificial hand or awheelchair. In addition, the target object may be a tool to be used foranimals such as a collar or a horseshoe.

The sensor device 10 transmits information indicating a measurementresult (hereinafter, also referred to as sensor information), to aninformation processing apparatus 30. The transmission may be performedin real time concurrently with the measurement, or the information maybe stored and transmitted at an arbitrary timing after the measurement.

The sensor attachment apparatus 20 is an apparatus for fixing the sensordevice 10 to a target object. As illustrated in FIG. 1, the sensorattachment apparatus 20 has one or more attachment positions (the sensorattachment tool 21) for removably attaching the sensor devices 10, andthe sensor devices 10 can be attached to a part of or all of theattachment positions. The sensor attachment apparatus 20 may be formedinto a shape that covers a part of or all of the trunk, the limbs, orthe like of a user, and in that case, it is desirable to form the sensorattachment apparatus 20 with extendable and retractable materials sothat a move of a user is not disturbed. In addition, the attached sensordevice 10 may be separated from the target object, and the sensorattachment apparatus 20 may have thickness like a helmet, a protector,and the like do. Additionally, the sensor attachment apparatus 20 may beattached to or be integrally formed with an object such as a golf club,a tennis racket, and a ski board. A user can attach the sensor device 10to the sensor attachment tool 21 positioned in a region that the userwants to measure.

The information processing apparatus 30 acquires the sensor informationfrom the sensor device 10, and performs various kinds of processing forvisualizing an operation of a target object.

<1.2. Configuration Example of Sensor Device>

FIG. 2 is a block diagram illustrating an example of a configuration ofthe sensor device 10 according to the present embodiment. As illustratedin FIG. 2, the sensor device 10 according to the present embodimentincludes an inertial sensor 110, a communication section 120, a storagesection 130, and a control section 140.

(1) Inertial Sensor 110

The inertial sensor 110 is a device that performs measurement usinginertia. The inertial sensor 110 includes an acceleration sensor, a gyrosensor, a geomagnetic sensor, and the like, and outputs the measuredsensor information (e.g. acceleration and angular speed) to the controlsection 140.

(2) Communication Section 120

The communication section 120 is a communication module for performingtransmission and reception of data between itself and the informationprocessing apparatus 30 in a wired/wireless manner. The communicationsection 120 can perform communication conforming to an arbitrarycommunication method such as a Local Area Network (LAN), a wireless LAN,Wi-Fi (registered trademark), Bluetooth (registered trademark), orinfrared communication, for example. The communication section 120transmits the sensor information measured by the inertial sensor 110, tothe information processing apparatus 30.

(3) Storage Section 130

The storage section 130 temporarily or permanently stores programs andvarious types of data for operations of the sensor device 10. Forexample, the storage section 130 temporarily stores the informationmeasured by the inertial sensor 110.

(4) Control Section 140

The control section 140 corresponds to a Central Processing Unit (CPU),a Digital Signal Processor (DSP), or the like, and performs processingfor providing various types of functions of the sensor device 10. Thesensor device 10 operates on the basis of control performed by thecontrol section 140. The operation of the sensor device 10 that is basedon the control performed by the control section 140 will be described indetail later.

<1.3. Configuration Example of Information Processing Apparatus>

FIG. 3 is a block diagram illustrating an example of a configuration ofthe information processing apparatus 30 according to the presentembodiment. As illustrated in FIG. 3, the information processingapparatus 30 according to the present embodiment includes acommunication section 310, an input section 320, an output section 330,a storage section 340, and a control section 350.

(1) Communication Section 310

The communication section 310 is a communication module for performingtransmission and reception of data between itself and the sensor device10 in a wired/wireless manner. The communication section 310 can performcommunication conforming to an arbitrary communication method such as aLAN, a wireless LAN, Wi-Fi, Bluetooth, or infrared communication, forexample. The communication section 310 receives the sensor informationfrom the sensor device 10.

(2) Input Section 320

The input section 320 receives an input of information. For example, theinput section 320 receives an input of information from the user. Theinput section 320 outputs the input information to the control section350.

(3) Output Section 330

The output section 330 performs an output of information. For example,the output section 330 outputs information using an image, a sound,vibration, and/or the like. The output section 330 outputs informationon the basis of control performed by the control section 350.

(4) Storage Section 340

The storage section 340 temporarily or permanently stores programs andvarious types of data for operations of the information processingapparatus 30. For example, the storage section 340 stores an operationmodel to be described later.

(5) Control Section 350

The control section 350 corresponds to a CPU, a DSP, or the like, andperforms processing for providing various types of functions of theinformation processing apparatus 30. The control section 350 may beregarded as at least one electrical circuit formed so as to be able toexecute functional units disclosed in FIG. 3. As illustrated in FIG. 3,the control section 350 includes a model learning section 351, a modelacquisition section 352, a first position information calculationsection 353, a second position information calculation section 354, andan output control section 355. Note that the control section 350 canfurther include structural elements other than these structuralelements. In other words, the control section 350 can also performoperations other than operations of these structural elements. Theoperations of these structural elements will be described in detaillater.

<1.4. Technical Feature>

(1) Target Object

In the present embodiment, a target object is a real object. Inaddition, position information in the present embodiment is athree-dimensional coordinate (i.e. X-coordinate, Y-coordinate, andZ-coordinate) in a real space.

For example, the target object includes one or more moving objects. Inother words, the target object may be singular or plural. For example, ahuman and an object (e.g. tool) manipulated by a human can be the targetobject. The operation of the target object is represented by a timeseries variation of the attitude of the target object.

For example, the target object may include a moving object having aplurality of joints. Examples of such a target object include a human, arobot, and the like. In addition, the operation of the target object mayinclude time series variations of position information pieces of theplurality of joints of the moving object. In other words, theinformation processing apparatus 30 can visualize a complicatedoperation in which a positional relationship between the joints varies.

In the present embodiment, a target object to be visualized is assumedto be a human. Hereinafter, a human serving as a target object will alsobe referred to as a user.

The sensor device 10 is arranged in an arbitrary region of the targetobject. For example, the sensor device 10 may be arranged at a joint ofthe user. In this case, the sensor attachment tool 21 is arranged at thejoint of the user. A region in which the sensor device 10 is to bearranged, such as a position of the sensor attachment tool 21 to whichthe sensor device 10 is to be attached, for example, will also bereferred to as a first point. On the other hand, a region in which thesensor device 10 is not arranged, such as a position of the sensorattachment tool 21 to which the sensor device 10 is not attached, forexample, will also be referred to as a second point.

A manipulator of the information processing apparatus 30 and the usermay be the same person, or may be different persons. As an example, thefollowing description will be given assuming that the manipulator andthe user are the same person.

(2) Calculation of Position Information that is Based on SensorInformation

On the basis of sensor information measured by one or more sensordevices 10 arranged on the target object, the information processingapparatus 30 (e.g. the first position information calculation section353) identifies a time series variation of position information of theone or more sensor devices 10. More briefly, the information processingapparatus 30 calculates position information of the first point on thebasis of the sensor information. For example, the information processingapparatus 30 calculates the position information of the first point fromthe sensor information acquired from the sensor device 10, using aninertial navigation system (INS).

The inertial navigation system is a technology of calculating a sensorposition by integrating angular speed and acceleration a plurality oftimes, and is employed in ships, airplanes, or the like, for example. Inthe inertial navigation system, first of all, by integrating angularspeed (performing first integration), an attitude (i.e. an attitudeangle in the real space) of a sensor device is calculated. Subsequently,by integrating acceleration (performing second integration), speed ofthe sensor device is calculated. Next, by integrating speed (performingthird integration), a moving distance of the sensor device iscalculated. Then, by combining vectors of moving distances and attitudes(i.e. moving directions) for each subdivision point, relative positioninformation starting from an initial position is calculated. If theinitial position is already known, absolute position information (i.e. athree-dimensional coordinate in the real space) of the sensor device canbe calculated by the above calculation.

For example, an example of measuring a state in which the user swings agolf club called an iron will be assumed. As an example, the sensordevices 10 are assumed to be attached to a neck, an waist, a right knee,a right foot, a left knee, a left foot, a hand, and a club head of theiron. In this case, the information processing apparatus 30 canvisualize an operation in which the user swings the iron, by calculatingposition information of each of the sensor devices 10. The example ofthe visualization will be described with reference to FIG. 4.

FIG. 4 is a diagram illustrating an example of visualization processingof a user operation according to the present embodiment. Attitudes 40Ato 40F indicate attitudes (i.e. position information pieces of therespective sensor devices 10) at the respective timings of the user thatswings the iron, and time flows in order from the attitude 40A towardthe attitude 40F. The attitude 40A indicates an attitude at a timing atwhich the user holds the iron at the ready. The attitude 40B indicatesan attitude at a timing of a back swing. The attitude 40C indicates anattitude at a timing at which the back swing has reached the top. Theattitude 40D indicates an attitude at a timing of a down swing. Theattitude 40E indicates an attitude at a timing of follow through. Theattitude 40F indicates an attitude at a timing of finish. Plots 41 ineach attitude indicate positions to which the sensor devices 10 areattached. A plot 41A corresponds to a neck, a plot 41B corresponds to anwaist, a plot 41C corresponds to a right knee, a plot 41D corresponds toa right foot, a plot 41E corresponds to a left knee, a plot 41Fcorresponds to a left foot, a plot 41G corresponds to a hand, and a plot41H corresponds to a club head of the iron.

Aside from the inertial navigation system, the information processingapparatus 30 can calculate the position information of the first pointusing an arbitrary algorithm. For example, by an optical motion capturetechnology that uses a captured image of a marker provided on a partialjoint of the user, the information processing apparatus 30 may calculateposition information of the joint to which the marker is added.

(3) Operation Model

An operation model is information indicating an operation pattern of amodeled object related to a target object.

The modeled object is an object to which the sensor device 10 isattached. The target object and the modeled object may be the same, ormay be different. For example, the user and a person modeled as amodeled object may be the same person, or may be different persons.

The operation model differs depending on the context, and is stored inthe storage section 340 for each context. In other words, the operationmodel can include a plurality of operation models corresponding tocontexts, such as a first operation model corresponding to a firstcontext, and a second operation model corresponding to a second context.The operation model is information indicating a time series variation ofposition information of each region of a modeled object that isobtainable in a case where the information processing apparatus 30operates in a context in which there is a modeled object. As an example,an operation model related to a swing operation of the iron will bedescribed. In this case, the operation model includes informationindicating time series variations of position information pieces of therespective regions of the neck, the waist, the right knee, the rightfoot, the left knee, the left foot, the hand, and the club head of theiron, for example. Here, the respective regions corresponding to theposition information pieces included in the operation model correspondto regions in which the sensor devices 10 are arranged (i.e. positionsin which the sensor attachment tool 21 are arranged).

Various types of contexts can be considered. For example, the contextscan include the type of the modeled object, the type of an operationperformed by the modeled object, attribute information of the modeledobject, and information indicating a state. Specifically, for example,in a case where the modeled object is a person, the contexts can includethe type of an operation such as walking, running, golf, and tennis,attribute information such as gender, age, a body height, and a bodyweight, information indicating a health state, a habit of an operation,etc., and the like.

For example, the operation model can be represented as a regressionmodel. In addition, the operation model may be represented in a formatobtained by dimension-compressing multidimensional information.

In addition, the information processing apparatus 30 (the model learningsection 351) may learn an operation model. For example, as for a swingoperation of the iron, the information processing apparatus 30 generatesan operation model on the basis of calculation of position informationthat is based on sensor information obtainable in a case where the userperforms a swing operation in a state in which the sensor devices 10 areattached to all the sensor attachment tools 21 and the club head of theiron.

(4) Selection of Operation Model

The information processing apparatus 30 (e.g. the model acquisitionsection 352) acquires at least one operation model related to a targetobject. Specifically, the information processing apparatus 30 acquires,from among operation models stored in the storage section 340, anoperation model corresponding to the context of the target object. Forexample, as for a swing operation of the iron, the informationprocessing apparatus 30 acquires an operation model in which a modeledobject is a person, and the type of an operation is golf.

For example, the information processing apparatus 30 may acquire anoperation model on the basis of a user manipulation. For example, thecontext of the user is input to the information processing apparatus 30by the user itself, and an operation model corresponding to the inputcontext is selected. In this case, it becomes possible to correctlyrefer to an operation model corresponding to the context designated bythe user. Note that the user manipulation includes both of anintentionally-performed manipulation (e.g. manual manipulation) and anunconsciously-performed manipulation (e.g. image recognition of anoperation).

For example, the information processing apparatus 30 may acquire anoperation model corresponding to an identified partial operation ofoperations of the target object. Therefore, first of all, theinformation processing apparatus 30 recognizes a context on the basis ofa partial operation of the user that has been identified using thesensor information measured by the sensor device 10 arranged on theuser. In other words, the information processing apparatus 30 recognizesa context of the user on the basis of the time series variation of thecalculated position information of the first point. Subsequently, theinformation processing apparatus 30 automatically acquires, from thestorage section 340, an operation model corresponding to the recognizedcontext. For example, the information processing apparatus 30automatically acquires one of the first operation model and the secondoperation model in accordance with the recognized context. In this case,it becomes possible for the information processing apparatus 30 toreduce the burden of a context input.

Note that the information processing apparatus 30 may acquire the firstoperation model and the second operation model at different timings. Inthis case, the information processing apparatus 30 can change anoperation model to be referred to, in accordance with a change in thecontext of the user. In addition, the information processing apparatus30 may simultaneously acquire the first operation model and the secondoperation model. In this case, it becomes possible for the informationprocessing apparatus 30 to calculate an operation of the target objectby combining a plurality of operation models.

(5) Calculation of Position Information that is Based on Operation Model

The information processing apparatus 30 (e.g. the second positioninformation calculation section 354) calculates an overall operationincluding the identified partial operation of the operations of thetarget object, with reference to the acquired operation model. Becausethe information processing apparatus 30 can calculate the overalloperation even in a case where only a part of the operations of thetarget object is identified, the operation of the target object can bevisualized from a smaller amount of information. Note that, as thenumber of first points increases, the accuracy of visualization isenhanced.

More specifically, the information processing apparatus 30 calculates atime series variation of position information of another region of thetarget object that follows an operation pattern indicated by anoperation model, and corresponds to a time series variation of positioninformation of one or more sensor devices. In other words, theinformation processing apparatus 30 calculates a time series variationof position information of the second point that follows an operationpattern indicated by an operation model, and corresponds to a timeseries variation of position information of the first point. It therebybecomes possible for the information processing apparatus 30 tocalculate an overall operation of the user that includes time seriesvariations of the position information of the first point and theposition information of the second point. Note that, hereinafter,calculating the position information of the second point will also bereferred to as predicting.

For example, the information processing apparatus 30 may calculate anoverall operation on the basis of an identified partial operation of theoperations of the user, with reference to an operation model. The targetobject may be a plurality of objects, and the information processingapparatus 30 may calculate operations of the objects manipulated by theuser, on the basis of a part or all of operations of the user that havebeen identified, with reference to an operation model. For example, theinformation processing apparatus 30 calculates an operation of a golfclub swung by the user, on the basis of an identified swing operation ofthe user, with reference to an operation model related to a swingoperation of a golf club. It thereby becomes possible to calculate thetrajectory of the golf club even if the sensor device 10 is not attachedto the golf club.

The following assumption will be given of an example of measuring astate in which the user swings an iron, similarly to the exampleillustrated in FIG. 4. Nevertheless, unlike the example illustrated inFIG. 4, the sensor devices 10 are assumed to be attached only to theright knee and the hand. The example of the visualization will bedescribed with reference to FIG. 5 and FIG. 6.

FIG. 5 is a diagram illustrating an example of visualization processingof a user operation according to the present embodiment. First of all,the information processing apparatus 30 (e.g. the first positioninformation calculation section 353) calculates position informationpieces of first points (i.e. the right knee and the hand) to which thesensor devices 10 are attached. An attitude 42 indicates a part ofattitudes of the user that is measured at a timing of a back swing, anda plot 43C corresponds to the right knee and a plot 43G corresponds tothe hand. Subsequently, the information processing apparatus 30 (e.g.the model acquisition section 352) selects, as an operation modelcorresponding to a context of the user, a golf operation model 44B fromamong a walking operation model 44A, the golf operation model 44B, and atennis operation model 44C. Then, the information processing apparatus30 (e.g. the second position information calculation section 354)predicts position information pieces of second points (i.e. the neck,the waist, the right foot, the left knee, the left foot, and the clubhead of the iron), from the position information pieces of the firstpoints, with reference to the golf operation model 44B. An attitude 45indicates an attitude obtained by adding prediction results of otherattitudes to the part of the attitudes of the user that is measured atthe timing of the back swing. A plot 43A indicates the neck, a plot 43Bindicates the waist, a plot 43D indicates the right foot, a plot 43Eindicates the left knee, a plot 43F indicates the left foot, and a plot43H indicates the club head of the iron.

By performing the above-described prediction processing at each timingof the swing, the information processing apparatus 30 can visualize theentire operation in which the user swings the iron, as illustrated inFIG. 6.

FIG. 6 is a diagram illustrating an example of visualization processingof a user operation according to the present embodiment. Attitudes 46Ato 46F indicate positions (plots 43C and 43G) of the sensor devices 10,and prediction results (plots 43A, 43B, 43D, 43E, 43F, and 43H) ofpositions of the other regions, at the respective timings of the userthat performs a swing operation. Time flows in order from the attitude46A toward the attitude 46F, and the timings from the attitude 46A tothe attitude 46F are similar to the respective timings from the attitude40A to the attitude 40F in FIG. 4. In addition, the attitude 45illustrated in FIG. 5 corresponds to the attitude 46B in FIG. 6.

In this manner, by predicting position information pieces of the regionsto which the sensor devices 10 are not attached, from the positioninformation pieces of the sensor devices 10, the information processingapparatus 30 can visualize the entire swing operation of the user.

Here, it is desirable that the sensor device 10 is arranged at a jointcorresponding to a context, among a plurality of predefined joints. Thisis because the arrangement position that can assure prediction accuracycan vary depending on the context. More specifically, as describedlater, it is desirable that the sensor device 10 is arranged at a jointcorresponding to a factor having a large factor loading that isindicated by a principal component result in each context. For example,in a case where a context is a swing operation of a golf club, it isdesirable that the sensor devices 10 are arranged at two locationsincluding the hand and the knee (the right knee in the case of righthandedness). It thereby becomes possible to predict a swing operationusing a smaller number of sensor devices 10.

(6) Prediction Algorithm

Various types of prediction algorithms of position information pieces ofsecond points can be considered.

Linear Regression

For example, using a linear regression model, the information processingapparatus 30 (e.g. the second position information calculation section354) may predict position information pieces of second points bytreating position information pieces of first points as inputs. In thiscase, an operation model is the linear regression model.

FIG. 7 is a diagram for describing an example of a prediction algorithmaccording to the present embodiment. FIG. 7 represents time seriesvariations of position information pieces of the iron, the left knee,the left foot, the right foot, the neck, and the waist that are to beobtained in a case where position information pieces of the right kneeand the hand are input to the regression model. A horizontal axis ofeach graph illustrated in FIG. 7 indicates a time, and a vertical axisindicates an X-coordinate, a Y-coordinate, or a Z-coordinate. Inaddition, a broken line in each graph indicates a time series variationof position information at the time of learning, and a solid lineindicates a time series variation of position information at the time ofprediction.

Note that, in a case where the linear regression model is constructed asa probability model, the information processing apparatus 30 may predictposition information pieces of second points by the Bayes' estimation.The method of linear regression is not limited to the Bayes' estimation,and various kinds of models can be used.

Dimension Compression

For example, the information processing apparatus 30 (e.g. the secondposition information calculation section 354) may predict positioninformation pieces of second points using a dimension-compressedoperation model.

For example, the above-described swing operation of the iron isrepresented by three pieces of information including an X-coordinate, aY-coordinate, and a Z-coordinate of each of eight regions including theneck, the waist, the right knee, the right foot, the left knee, the leftfoot, the hand, and the club head of the iron, that is to say,represented by 24-dimensional information in total. Thedimension-compressed operation model represents the 24-dimensionalinformation as information in the dimension having a dimension numbersmaller than 24. For example, Principal Component Analysis (PCA) can beused for dimension compression.

Because there is a strong high-speed condition for the attitude of aperson, it is generally known that a multidimensional space representedby attitude parameters (i.e. position information pieces of therespective joints) having a multi-degree of freedom can be representedin a lower-dimensional space. Treating the attitude of a person in alow-dimensional space can be said to be equal to subconsciouslyconsidering a skeleton model.

An example of the dimension-compressed operation model will be describedbelow with reference to FIG. 8.

FIG. 8 is a diagram illustrating an example of a dimension-compressedoperation model according to the present embodiment. FIG. 8 illustratesan example of an operation model obtained by compressing 21-dimensionalinformation related to seven regions other than the club head of theiron, into three dimension including a first principal component (PC1),a second principal component (PC2), and a third principal component(PC3). The club head of the iron is excluded for enhancing predictionaccuracy. Each plot in a space of the dimension-compressed operationmodel corresponds to an attitude of a person at each timing in the golfswing. For example, plots 50A to 50F respectively correspond toattitudes 51A to 51F.

Subsequently, a prediction algorithm that uses a dimension-compressedoperation model will be described with reference to FIG. 9.

FIG. 9 is a diagram for describing an example of a prediction algorithmaccording to the present embodiment. As illustrated in FIG. 9, theinformation processing apparatus 30 calculates position informationpieces of first points (the right knee and the hand). Subsequently, theinformation processing apparatus 30 searches for a point on acompression space in which the position information pieces of the firstpoints are to be reproduced, by a steepest descent method (referencenumeral 52), and calculates position information pieces of the firstpoints and second points, from the retrieved one point, by PCA inversemapping (reference numeral 53). The information processing apparatus 30repeatedly performs these calculations until a difference between theposition information pieces of the first points that are obtainablebefore and after the search falls below a threshold value (referencenumeral 54). After that, the information processing apparatus 30predicts, by linear regression, position information of the club head ofthe iron on the basis of the position information pieces of the firstpoints and the second points (reference numeral 55).

By performing the above-described prediction processing at each timingof the swing, the information processing apparatus 30 can visualize theentire operation in which the user swings the iron, as illustrated inFIG. 6.

Note that the information processing apparatus 30 may use GaussianProcess Latent Variable Models (GPLVM), for example, as an alternativedimension compression method of the PCA. The GPLVM is a nonlinearcompression method, and is suitable for predicting a more complicatedoperation although costs of learning and prediction are higher thanthose in the PCA.

(7) Information Output

The information processing apparatus 30 (e.g. the output control section355) outputs output information indicating the calculated overalloperation of the target object. The information output may be performedin real time concurrently with the operation of the user, or may beperformed at an arbitrary timing after the measurement.

For example, the output information may be an image (a moving image or astill image) in which a virtual object corresponding to the targetobject performs an operation corresponding to the overall operation ofthe target object. For example, the output information may be an imageindicating a time series variation of position information of each pointof the user, as illustrated in FIG. 6. Additionally, the outputinformation may be an image in which an avatar corresponding to the userperforms an operation corresponding to an operation of the user. A UserInterface (UI) example in this case will be described with reference toFIG. 10.

FIG. 10 is a diagram for describing an example of a UI according to thepresent embodiment. As illustrated in FIG. 10, the user holds acontroller 60 for games that includes sensor devices 10A and 10B, andwears a virtual reality (VR) device 61. By varying a position or anattitude of the controller 60, the user manipulates an avatar displayedon the VR device 61. Specifically, the entire attitude of the user iscalculated on the basis of calculation results of position informationpieces of the sensor devices 10A and 10B, and a calculation result ofthe overall attitude is reflected on a move of the avatar. For example,as illustrated in FIG. 10, when the user holds up the controller 60overhead, and then, swings down the controller 60, an image 62A in whichthe avatar holds up a sword overhead, and an image 62B in which theavatar swings down the sword are displayed on the VR device 61. Here,although the sensor devices 10 are not arranged on the feet of the user,a move of the feet can be reproduced in the avatar by theabove-described prediction processing.

Additionally, the output information may be an instruction related to anattachment position of the sensor device 10. For example, theinformation processing apparatus 30 may output information instructingan optimum attachment position of the sensor device 10, on the basis ofan analysis result of PCA. Table 1 described below indicates an exampleof a contribution ratio and a factor loading of each principal componentin a swing operation of an iron.

TABLE 1 Principal component PC1 PC2 PC3 Contribution 0.6584 (65.8%)0.2820 (94.0%) 0.0500 (99.0%) ratio (cumulative) Factor Hand (0.0918)Hand (0.0296) Hand (0.0059) loading Right (0.0237) Right (0.0154) Right(0.0041) Knee Knee Knee Waist (0.0152) Left (0.0129) Left (0.0025) KneeKnee Right (0.0080) Waist (0.0054) Waist (0.0014) Foot Left (0.0072)Right (0.0043) Right (0.0014) Knee Foot Foot Neck (0.0070) Neck (0.0037)Neck (0.0012) Left (0.0010) Left (0.0005) Left (0.0003) Foot Foot Foot

As illustrated in Table 1 described above, a contribution ratio of afirst principal component is 65.8%, a cumulative contribution ratio ofthe first principal component and a second principal component is 94%,and a cumulative contribution ratio of first to third principalcomponents is 99%. In addition, in Table 1 described above, factors arelisted in the order of larger factor loadings of each principalcomponent. When the sensor devices 10 are attached to regionscorresponding to factors having larger sums of factor loadings in thefirst to third principal components, it becomes possible to predict aswing operation using a smaller number of sensor devices 10. Forexample, according to Table 1 described above, in a case where thenumber of sensor devices 10 is two, the sensor devices 10 are desirablyattached to the hand and the right knee. In view of the foregoing, theinformation processing apparatus 30 outputs information instructing theuser that performs a swing operation, to attach the sensor devices 10 tothe hand and the right knee.

<1.5. Flow of Processing>

Subsequently, an example of a flow of operation visualization processingexecuted in the information processing apparatus 30 according to thepresent embodiment will be described with reference to FIG. 11. FIG. 11is a flowchart illustrating an example of a flow of operationvisualization processing executed in the information processingapparatus 30 according to the present embodiment.

As illustrated in FIG. 11, first of all, the information processingapparatus 30 acquires an operation model corresponding to a context(step S102). For example, the information processing apparatus 30acquires an operation model on the basis of a user manipulation, oracquires an operation model corresponding to an identified partialoperation of the operations of a target object. Subsequently, theinformation processing apparatus 30 acquires sensor information measuredand transmitted by the sensor device 10 (step S104). Subsequently, theinformation processing apparatus 30 calculates position information of afirst point on the basis of the sensor information (step S106). Then,the information processing apparatus 30 calculates position informationof a second point on the basis of the position information of the firstpoint and the operation model (step S108). Next, the informationprocessing apparatus 30 generates output information (step S110). Forexample, the information processing apparatus 30 generates an image onthe basis of the position information of the first point and theposition information of the second point, or generates informationinstructing an optimum attachment position of the sensor device 10, onthe basis of an analysis result of PCA. Subsequently, the informationprocessing apparatus 30 outputs the generated output information (stepS112).

The processing ends through the above flow.

2. Second Embodiment

The present embodiment is a mode of visualizing an operation of avirtual object.

A system 1 according to the present embodiment includes an informationprocessing apparatus 30. A configuration example of the informationprocessing apparatus 30 according to the present embodiment will bedescribed below with reference to FIG. 12.

<2.1. Configuration Example of Information Processing Apparatus>

FIG. 12 is a block diagram illustrating an example of a configuration ofthe information processing apparatus 30 according to the presentembodiment. As illustrated in FIG. 12, the information processingapparatus 30 according to the present embodiment includes the inputsection 320, the output section 330, the storage section 340, and acontrol section 350. Note that, because the functions of the inputsection 320, the output section 330, and the storage section 340 aresimilar to those in the first embodiment, the description here will beomitted.

The control section 350 corresponds to a CPU, a DSP, or the like, andperforms processing for providing various types of functions of theinformation processing apparatus 30. The control section 350 may beregarded as at least one electrical circuit formed so as to be able toexecute functional units disclosed in FIG. 12. As illustrated in FIG.12, the control section 350 includes the model acquisition section 352,the first position information calculation section 353, the secondposition information calculation section 354, a keyframe registrationsection 356, and a complementing section 357. Note that the controlsection 350 can further include structural elements other than thesestructural elements. In other words, the control section 350 can alsoperform operations other than operations of these structural elements.The operations of these structural elements will be described in detaillater.

<2.2. Technical Feature>

(1) Target Object

In the present embodiment, a target object is a virtual object. Inaddition, position information in the present embodiment is athree-dimensional coordinate in a virtual space.

For example, the target object may include one or more virtual movingobjects.

For example, the target object may include a virtual moving objecthaving a plurality of joints. Then, an operation of the target objectmay include time series variations of position information pieces of theplurality of joints of the virtual moving object.

In the present embodiment, a target object to be visualized is assumedto be an avatar of a virtual human.

Hereinafter, a manipulator of the information processing apparatus 30will also be referred to as a user.

In the present embodiment, among regions of a virtual object, a regionmanipulated by the user will also be referred to as a first point. Inaddition, among regions of a virtual object, a region not manipulated bythe user will also be referred to as a second point.

(2) Calculation of Position Information that is Based on UserManipulation

The information processing apparatus 30 (e.g. the first positioninformation calculation section 353) identifies a partial operation ofthe virtual object on the basis of manipulation information of the userwith respect to the virtual object. Specifically, the informationprocessing apparatus 30 calculates position information of the firstpoint on the basis of manipulation information indicating a usermanipulation instructing a partial operation of the virtual object Forexample, upon receiving a manipulation of moving one joint of an avatarby drag, the information processing apparatus 30 calculates positioninformation of the moved joint on the basis of a drag amount and a dragdirection.

(3) Calculation of Position Information that is Based on Operation Model

The information processing apparatus 30 (e.g. the second positioninformation calculation section 354) calculates a time series variationof position information of another region of the target object thatfollows an operation pattern indicated by an operation model, andcorresponds to a time series variation of position information of one ormore manipulation target regions. In other words, the informationprocessing apparatus 30 calculates a time series variation of positioninformation of the second point that follows an operation patternindicated by an operation model, and corresponds to a time seriesvariation of position information of the first point. It thereby becomespossible for the information processing apparatus 30 to calculate anoverall operation of the avatar that includes time series variations ofthe position information of the first point and the position informationof the second point.

Note that the technical features related to an operation model, theselection of the operation model, and prediction algorithms are similarto those in the first embodiment except that the target object is avirtual object, and calculation of position information that is based onsensor information becomes calculation of position information that isbased on manipulation information.

(4) Information Output

UI Example

Typically, the information processing apparatus 30 according to thepresent embodiment is used for production support for 3D animation. Anexample of a UI for production support for 3D animation will bedescribed with reference to FIG. 13.

FIG. 13 is a diagram illustrating an example of a UI according to thepresent embodiment. As illustrated in FIG. 13, an avatar 71 is displayedon a production screen 70A. On the production screen 70A, the userselects and drags a right hand 72 of the avatar 71 by a pointer 73. As aresult, as in a production screen 70B, not only the dragged right hand72 (i.e. the first point), but also a left hand, both feet, and the like(i.e. second points) move in accordance with the move of the right hand72. In this manner, the user can cause the avatar 71 to perform anatural operation, only by designating a move of a partial region,without finely designating a whole-body move of the avatar 71 for eachregion. Accordingly, load on production of 3D animation can be reduced.

Generation of Animation

The information processing apparatus 30 generates animation using akeyframe method, for example. The keyframe method is a technology ofgenerating a moving image by arranging keyframes at every severalframes, and complementing between the keyframes.

For example, the information processing apparatus 30 (e.g. the keyframeregistration section 356) registers a keyframe of animation. Forexample, the information processing apparatus 30 registers, as akeyframe, an attitude of the avatar 71 manipulated by the user on aproduction screen as illustrated in FIG. 13.

Subsequently, the information processing apparatus 30 (e.g. thecomplementing section 357) generates animation by complementing betweenkeyframes.

<2.3. Flow of Processing>

Subsequently, an example of a flow of operation visualization processingexecuted in the information processing apparatus 30 according to thepresent embodiment will be described with reference to FIG. 14. FIG. 14is a flowchart illustrating an example of a flow of operationvisualization processing executed in the information processingapparatus 30 according to the present embodiment.

As illustrated in FIG. 14, first of all, the information processingapparatus 30 acquires an operation model corresponding to a context(step S202). For example, the information processing apparatus 30acquires an operation model on the basis of a user manipulation.Subsequently, the information processing apparatus 30 acquiresmanipulation information indicating a user manipulation instructing apartial operation of the virtual object (step S204). Next, theinformation processing apparatus 30 calculates position information ofthe first point on the basis of the manipulation information (stepS206). Then, the information processing apparatus 30 calculates positioninformation of the second point on the basis of the position informationof the first point and the operation model (step S208). Next, theinformation processing apparatus 30 generates animation (step S210). Forexample, the information processing apparatus 30 generates animation byregistering a group of keyframes, and complementing between thekeyframes. Subsequently, the information processing apparatus 30 outputsthe generated animation (step S212).

The processing ends through the above flow.

3. Hardware Configuration Example

Finally, a hardware configuration of an information processing apparatusaccording to each of the embodiments will be described with reference toFIG. 15. FIG. 15 is a block diagram illustrating an example of thehardware configuration of the information processing apparatus accordingto each of the embodiments. Meanwhile, the information processingapparatus 900 illustrated in FIG. 15 may realize the informationprocessing apparatus 30 illustrated in each of FIG. 3 or FIG. 12, forexample. Information processing by the information processing apparatus30 according to each of the embodiments is realized according tocooperation between software and hardware described below.

As illustrated in FIG. 15, the information processing apparatus 900includes a central processing unit (CPU) 901, a read only memory (ROM)902, a random access memory (RAM) 903 and a host bus 904 a. In addition,the information processing apparatus 900 includes a bridge 904, anexternal bus 904 b, an interface 905, an input device 906, an outputdevice 907, a storage device 908, a drive 909, a connection port 911 anda communication device 913. The information processing apparatus 900 mayinclude a processing circuit such as a DSP or an ASIC instead of the CPU901 or along therewith.

The CPU 901 functions as an arithmetic processing device and a controldevice and controls the overall operation in the information processingapparatus 900 according to various programs. Further, the CPU 901 may bea microprocessor. The ROM 902 stores programs, operation parameters andthe like used by the CPU 901. The RAM 903 temporarily stores programsused in execution of the CPU 901, parameters appropriately changed inthe execution, and the like. The CPU 901 may form the control section350 illustrated in FIG. 3 or FIG. 12, for example.

The CPU 901, the ROM 902 and the RAM 903 are connected by the host bus904 a including a CPU bus and the like. The host bus 904 a is connectedwith the external bus 904 b such as a peripheral componentinterconnect/interface (PCI) bus via the bridge 904. Further, the hostbus 904 a, the bridge 904 and the external bus 904 b are not necessarilyseparately configured and such functions may be mounted in a single bus.

The input device 906 is realized by a device through which a user inputsinformation, such as a mouse, a keyboard, a touch panel, a button, amicrophone, a switch, and a lever. In addition, the input device 906 maybe a remote control device using infrared ray or other electric waves orexternal connection equipment such as a cellular phone or a PDAcorresponding to operation of the information processing apparatus 900,for example. Furthermore, the input device 906 may include an inputcontrol circuit or the like which generates an input signal on the basisof information input by the user using the aforementioned input meansand outputs the input signal to the CPU 901, for example. The user ofthe information processing apparatus 900 may input various types of dataor order a processing operation for the information processing apparatus900 by operating the input device 906. The input device 906 may form theinput section 320 illustrated in FIG. 3 or FIG. 12, for example.

In addition to the above, the input device 906 can be formed by a devicethat detects information related to the user. For example, the inputdevice 906 can include various sensors such as an image sensor (acamera, for example), a depth sensor (a stereo camera, for example), anacceleration sensor, a gyro sensor, a geomagnetic sensor, an opticalsensor, a sound sensor, a distance measurement sensor, and a forcesensor. Also, the input device 906 may acquire information related tothe state of the information processing apparatus 900 itself such as theposture and the moving velocity of the information processing apparatus900 and information related to a surrounding environment of theinformation processing apparatus 900 such as brightness or noise aroundthe information processing apparatus 900. Also, the input device 906 mayinclude a GNSS module that receives a GNSS signal (a GPS signal from aglobal positioning system (GPS) satellite, for example) from a globalnavigation satellite system (GNSS) satellite and measures positioninformation including the latitude, the longitude, and the altitude ofthe device. In addition, the input device 906 may detect the positionthrough Wi-Fi (registered trademark), transmission and reception to andfrom a mobile phone, a PHS, a smartphone, or the like, near-fieldcommunication, or the like, in relation to the position information.

The output device 907 is formed by a device that may visually or aurallynotify the user of acquired information. As such devices, there is adisplay device such as a CRT display device, a liquid crystal displaydevice, a plasma display device, an EL display device, a laserprojector, an LED projector or a lamp, a sound output device such as aspeaker and a headphone, a printer device and the like. The outputdevice 907 outputs results acquired through various processes performedby the information processing apparatus 900, for example. Specifically,the display device visually displays results acquired through variousprocesses performed by the information processing apparatus 900 invarious forms such as text, images, tables and graphs. On the otherhand, the sound output device converts audio signals includingreproduced sound data, audio data and the like into analog signals andaurally outputs the analog signals. The aforementioned display deviceand the aforementioned sound output device may form the output section330 illustrated in FIG. 3 or FIG. 12, for example.

The storage device 908 is a device for data storage, formed as anexample of a storage section of the information processing apparatus900. For example, the storage device 908 is realized by a magneticstorage device such as an HDD, a semiconductor storage device, anoptical storage device, a magneto-optical storage device or the like.The storage device 908 may include a storage medium, a recording devicefor recording data on the storage medium, a reading device for readingdata from the storage medium, a deletion device for deleting datarecorded on the storage medium and the like. The storage device 908stores programs and various types of data executed by the CPU 901,various types of data acquired from the outside and the like. Thestorage device 908 may form the storage section 340 illustrated in FIG.3 or FIG. 12, for example.

The drive 909 is a reader/writer for storage media and is included in orexternally attached to the information processing apparatus 900. Thedrive 909 reads information recorded on a removable storage medium suchas a magnetic disc, an optical disc, a magneto-optical disc or asemiconductor memory mounted thereon and outputs the information to theRAM 903. In addition, the drive 909 can write information on theremovable storage medium.

The connection port 911 is an interface connected with externalequipment and is a connector to the external equipment through whichdata may be transmitted through a universal serial bus (USB) and thelike, for example.

The communication device 913 is a communication interface formed by acommunication device for connection to a network 920 or the like, forexample. The communication device 913 is a communication card or thelike for a wired or wireless local area network (LAN), long termevolution (LTE), Bluetooth (registered trademark) or wireless USB(WUSB), for example. In addition, the communication device 913 may be arouter for optical communication, a router for asymmetric digitalsubscriber line (ADSL), various communication modems or the like. Forexample, the communication device 913 may transmit/receive signals andthe like to/from the Internet and other communication apparatusesaccording to a predetermined protocol, for example, TCP/IP or the like.The communication device 913 may form the communication section 310illustrated in FIG. 3, for example.

Further, the network 920 is a wired or wireless transmission path ofinformation transmitted from devices connected to the network 920. Forexample, the network 920 may include a public circuit network such asthe Internet, a telephone circuit network or a satellite communicationnetwork, various local area networks (LANs) including Ethernet(registered trademark), a wide area network (WAN) and the like. Inaddition, the network 920 may include a dedicated circuit network suchas an internet protocol-virtual private network (IP-VPN).

Hereinbefore, an example of a hardware configuration capable ofrealizing the functions of the information processing apparatus 900according to each of the embodiments is shown. The respective componentsmay be implemented using universal members, or may be implemented byhardware specific to the functions of the respective components.Accordingly, according to a technical level at the time when each of theembodiments are executed, it is possible to appropriately changehardware configurations to be used.

In addition, a computer program for realizing each of the functions ofthe information processing apparatus 900 according to each of theembodiments as described above may be created, and may be mounted in aPC or the like. Furthermore, a computer-readable recording medium onwhich such a computer program is stored may be provided. The recordingmedium is a magnetic disc, an optical disc, a magneto-optical disc, aflash memory, or the like, for example. Further, the computer programmay be delivered through a network, for example, without using therecording medium.

4. Conclusion

An embodiment of the present disclosure has been described in detailabove with reference to FIG. 1 to FIG. 15. As described above, theinformation processing apparatus 30 according to the present embodimentacquires an operation model indicating an operation pattern related to atarget object, and calculates an overall operation from an identifiedpartial operation of operations of the target object, with reference tothe acquired operation model. In the first embodiment, it therebybecomes possible to predict operations of the overall regions of thetarget object, by sensor information pieces obtained from the sensordevices 10 attached to partial regions of the target object. Inaddition, in the second embodiment, it becomes possible to generateanimation in which the overall regions of the target object areoperated, by a user manipulation of operating a partial region of thetarget object. In this manner, the information processing apparatus 30can visualize an operation of a target object from a smaller amount ofinformation. Note that operation information may be output byspecifying, as a target object, a specific section such as an arm, afoot, an upper body, or a lower body of the user. In other words, inthis specification, a “target object” is not limited to a target objectthat is physically or virtually independent.

The preferred embodiment(s) of the present disclosure has/have beendescribed above with reference to the accompanying drawings, whilst thepresent disclosure is not limited to the above examples. A personskilled in the art may find various alterations and modifications withinthe scope of the appended claims, and it should be understood that theywill naturally come under the technical scope of the present disclosure.

For example, the first embodiment and the second embodiment of thepresent disclosure can be appropriately combined. For example, animationindicating a user operation may be generated by registering keyframes onthe basis of operations of the user at respective timings, andcomplementing between the keyframes.

For example, devices described in this specification may be implementedas independent devices, or a part or all thereof may be implemented asseparate devices. For example, in the functional configuration exampleof the information processing apparatus 30 that is illustrated in FIG. 3or 12, the storage section 340 and/or the control section 350 may beincluded in a device such as a server that is connected with the inputsection 320 and the output section 330 via a network or the like. Inaddition, the sensor device 10 and the information processing apparatus30 may be integrally formed.

Note that it is not necessary for the processing described in thisspecification with reference to the flowchart and the sequence diagramto be executed in the order shown in the flowchart. Some processingsteps may be performed in parallel. Further, some of additional stepscan be adopted, or some processing steps can be omitted.

Further, the effects described in this specification are merelyillustrative or exemplified effects, and are not limitative. That is,with or in the place of the above effects, the technology according tothe present disclosure may achieve other effects that are clear to thoseskilled in the art from the description of this specification.

Additionally, the present technology may also be configured as below.

(1)

An information processing apparatus including:

an acquisition section configured to acquire an operation modelindicating an operation pattern related to a target object;

a calculation section configured to calculate an overall operationincluding an identified partial operation of operations of the targetobject, with reference to the operation model acquired by theacquisition section; and

an output control section configured to output output informationindicating the overall operation of the target object that has beencalculated by the calculation section.

(2)

The information processing apparatus according to (1), in which theacquisition section acquires the operation model corresponding to acontext of the target object.

(3)

The information processing apparatus according to (2), in which theacquisition section acquires the operation model on a basis of a usermanipulation.

(4)

The information processing apparatus according to (2), in which theacquisition section acquires the operation model corresponding to anidentified partial operation of operations of the target object.

(5)

The information processing apparatus according to (3) or (4), in whichthe target object includes a user,

the operation model includes a first operation model and a secondoperation model, and

the acquisition section recognizes the context on a basis of a partialoperation of the user that has been identified by sensor informationmeasured by a sensor device arranged on the user, and automaticallyacquires one of the first operation model and the second operation modelin accordance with the recognized context.

(6)

The information processing apparatus according to any one of (1) to (5),in which the output information includes an image in which a virtualobject corresponding to the target object performs an operationcorresponding to the overall operation of the target object.

(7)

The information processing apparatus according to (6), in which thetarget object includes a user, and the output information is displayedby a virtual reality (VR) device worn by the user.

(8)

The information processing apparatus according to any one of (1) to (7),in which the target object is a real object, and

the calculation section identifies, on a basis of sensor informationmeasured by a sensor device arranged on the target object, a time seriesvariation of position information of the sensor device.

(9)

The information processing apparatus according to (8), in which thecalculation section calculates a time series variation of positioninformation of another region of the target object that follows theoperation pattern indicated by the operation model, and corresponds to atime series variation of position information of the one or more sensordevices.

(10)

The information processing apparatus according to (8) or (9), in whichthe sensor device is arranged at a joint corresponding to a context,among a plurality of predefined joints.

(11)

The information processing apparatus according to (10), in which thesensor devices are arranged at two locations including a hand and aknee, in a case where the context is a swing operation of a golf club.

(12)

The information processing apparatus according to any one of (8) to(11), in which the sensor device includes an inertial sensor.

(13)

The information processing apparatus according to any one of (8) to(12), in which the sensor device is removably attached to a sensorattachment tool for fixing the sensor device on the target object.

(14)

The information processing apparatus according to (1), in which thetarget object includes a user and an object manipulated by the user.

(15)

The information processing apparatus according to (14), in which thecalculation section calculates, on a basis of an identified operation ofa user, an operation of an object manipulated by the user, withreference to the operation model.

(16)

The information processing apparatus according to (15), in which thecalculation section calculates, on a basis of an identified swingoperation of a user, an operation of a golf club swung by the user, withreference to the operation model related to a swing operation of a golfclub.

(17)

The information processing apparatus according to any one of (1) to (7),in which the target object is a virtual object, and

the calculation section identifies a partial operation of the virtualobject on a basis of manipulation information of a user with respect tothe virtual object.

(18)

The information processing apparatus according to any one of (1) to(17), in which the target object includes a moving object having aplurality of joints, and

an operation of the target object includes time series variations ofposition information of the plurality of joints of the moving object.

(19)

An information processing method including:

acquiring an operation model indicating an operation pattern related toa target object;

calculating, by a processor, an overall operation including anidentified partial operation of operations of the target object, withreference to the acquired operation model; and

outputting output information indicating the calculated overalloperation of the target object.

(20)

A storage medium storing a program for causing a computer to functionas:

an acquisition section configured to acquire an operation modelindicating an operation pattern related to a target object;

a calculation section configured to calculate an overall operationincluding an identified partial operation of operations of the targetobject, with reference to the operation model acquired by theacquisition section; and

an output control section configured to output output informationindicating the overall operation of the target object that has beencalculated by the calculation section.

REFERENCE SIGNS LIST

-   1 system-   10 sensor device-   110 inertial sensor-   120 communication section-   130 storage section-   140 control section-   20 sensor attachment apparatus-   21 sensor attachment tool-   30 information processing apparatus-   310 communication section-   320 input section-   330 output section-   340 storage section-   350 control section-   351 model learning section-   352 model acquisition section-   353 first position information calculation section-   354 second position information calculation section-   355 output control section-   356 keyframe registration section-   357 complementing section

The invention claimed is:
 1. An information processing apparatus,comprising: a memory configured to store a plurality of operationmodels; and circuitry configured to: acquire sensor informationassociated with a target object, wherein the acquired sensor informationincludes a first plurality of position information pieces, each positioninformation piece of the first plurality of position information piecescorresponds to a respective sensor device of a plurality of sensordevices attached to a first plurality of regions on the target object,and the target object includes a user; determine a partial operation ofthe user based on the acquired sensor information; determine a type ofthe partial operation of the target object based on the determinedpartial operation; acquire a specific operation model of the storedplurality of operation models, wherein the specific operation model isassociated with the type of the partial operation of the target object,the specific operation model corresponds to the determined partialoperation, and the specific operation model indicates an operationpattern associated with the target object; predict a second plurality ofposition information pieces of a second plurality of regions on thetarget object for each time point of the partial operation, wherein thesecond plurality of position information pieces of the second pluralityof regions on the target object is predicted based on the sensorinformation related to the first plurality of regions and the acquiredspecific operation model, the second plurality of regions is differentfrom the first plurality of regions, and the second plurality of regionsare regions on the target object to which no sensor devices areattached; determine an overall operation of the target object thatincludes the first plurality of regions and the second plurality ofregions; and output operation information that indicates the overalloperation of the target object.
 2. The information processing apparatusaccording to claim 1, wherein the circuitry is further configured toacquire the specific operation model based on a user manipulation. 3.The information processing apparatus according to claim 2, wherein thespecific operation model comprises a first operation model and a secondoperation model, and the circuitry is further configured to acquire atleast one of the first operation model or the second operation modelbased on the determined type of the partial operation of the targetobject.
 4. The information processing apparatus according to claim 1,wherein the operation information further comprises an image of avirtual object, and the virtual object corresponds to the target object.5. The information processing apparatus according to claim 4, whereinthe operation information is displayed by a virtual reality (VR) devicewearable by the user.
 6. The information processing apparatus accordingto claim 1, wherein the target object is a real object, the circuitry isfurther configured to calculate a time series variation of each positioninformation piece of the first plurality of position information pieces,and the time series variation of each position information piece of thefirst plurality of position information pieces is calculated based onthe sensor information.
 7. The information processing apparatusaccording to claim 6, wherein the circuitry is further configured tocalculate a time series variation of each position information piece ofthe second plurality of position information pieces, and the time seriesvariation of each position information piece of the second plurality ofposition information pieces is associated with the time series variationof each position information piece of the first plurality of positioninformation pieces.
 8. The information processing apparatus according toclaim 1, wherein a sensor device of the plurality of sensor devices isat a joint of a plurality of joints, and the plurality of joints isassociated with the type of the partial operation of the target object.9. The information processing apparatus according to claim 8, whereinthe sensor device is at one of two locations, the two locations are ahand of the user and a knee of the user, and the type of the partialoperation of the target object is a swing operation of a golf club. 10.The information processing apparatus according to claim 1, wherein asensor device of the plurality of sensor devices comprises an inertialsensor.
 11. The information processing apparatus according to claim 1,wherein a sensor device of the plurality of sensor devices is removablyattached to a sensor attachment tool.
 12. The information processingapparatus according to claim 1, wherein the target object furtherincludes an object manipulated by the user.
 13. The informationprocessing apparatus according to claim 12, wherein the circuitry isfurther configured to determine an operation of the object manipulatedby the user, and the operation of the object manipulated by the user isdetermined based on the partial operation and the specific operationmodel.
 14. The information processing apparatus according to claim 13,wherein the specific operation model corresponds to a swing operation ofa golf club, and the circuitry is further configured to determine anoperation of the golf club swung by the user based on the specificoperation model.
 15. The information processing apparatus according toclaim 1, wherein the target object is a virtual object, the circuitry isfurther configured to determine the partial operation of the virtualobject based on manipulation information, and the manipulationinformation is associated with manipulation of the virtual object by theuser.
 16. The information processing apparatus according to claim 1,wherein the target object is a moving object having a plurality ofjoints, and the operation of the target object includes time seriesvariations of position information of the plurality of joints of themoving object.
 17. An information processing method, comprising:storing, in a memory, a plurality of operation models; acquiring sensorinformation associated with a target object, wherein the acquired sensorinformation includes a first plurality of position information pieces,each position information piece of the first plurality of positioninformation pieces corresponds to a respective sensor device of aplurality of sensor devices attached to a first plurality of regions onthe target object, and the target object includes a user; determining apartial operation of the user based on the acquired sensor information;determining a type of the partial operation of the target object basedon the determined partial operation; acquiring a specific operationmodel of the stored plurality of operation models, wherein the specificoperation model is associated with the type of the partial operation ofthe target object, the specific operation model corresponds to thedetermined partial operation, and the specific operation model indicatesan operation pattern associated with the target object; predicting asecond plurality of position information pieces of a second plurality ofregions on the target object for each time point of the partialoperation, wherein the second plurality of position information piecesof the second plurality of regions on the target object is predictedbased on the sensor information related to the first plurality ofregions and the acquired specific operation model, the second pluralityof regions is different from the first plurality of regions, and thesecond plurality of regions are regions on the target object to which nosensor devices are attached; determining an overall operation of thetarget object that includes the first plurality of regions and thesecond plurality of regions; and outputting operation information thatindicates the overall operation of the target object.
 18. Anon-transitory computer-readable medium having stored thereon, computerexecutable-instructions that, when executed by a processor, cause theprocessor to execute operations, the operations comprising: storing, ina memory, a plurality of operation models; acquiring sensor informationassociated with a target object, wherein the acquired sensor informationincludes a first plurality of position information pieces, each positioninformation piece of the first plurality of position information piecescorresponds to a respective sensor device of a plurality of sensordevices attached to a first plurality of regions on the target object,and the target object includes a user; determining a partial operationof the user based on the acquired sensor information; determining a typeof the partial operation the target object based on the determinedpartial operation; acquiring a specific operation model of the storedplurality of operation models, wherein the specific operation model isassociated with the type of the partial operation of the target object,the specific operation model corresponds to the determined partialoperation, and the specific operation model indicates an operationpattern associated with the target object; predicting a second pluralityof position information pieces of a second plurality of regions on thetarget object for each time point of the partial operation, wherein thesecond plurality of position information pieces of the second pluralityof regions on the target object is predicted based on the sensorinformation related to the first plurality of regions and the acquiredspecific operation model, the second plurality of regions is differentfrom the first plurality of regions, and the second plurality of regionsare regions on the target object to which no sensor devices areattached; determining an overall operation of the target object thatincludes the first plurality of regions and the second plurality ofregions; and outputting operation information that indicates the overalloperation of the target object.